AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.18678	1.07
	2_DecisionTree	Decision Tree	logloss	0.0337236	11.04
the best	3_Default_Xgboost	Xgboost	logloss	0.00454643	8.93
	4_Default_NeuralNetwork	Neural Network	logloss	0.0172138	10.95
	5_Default_RandomForest	Random Forest	logloss	0.0107606	6.04
	Ensemble	Ensemble	logloss	0.00454643	0.61

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

0.7 seconds

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	0	0	0.511	0	0	0.511	0.1022	0.261121	1.18678
recall	0	0	1	0	0	0.511	0.2	0.511	1.18678
f1-score	0	0	0.676373	0	0	0.511	0.135275	0.345627	1.18678
support	4650	2820	8803	670	284	0.511	17227	17227	1.18678

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	0	0	4650	0	0
Labeled as far	0	0	2820	0	0
Labeled as nearby	0	0	8803	0	0
Labeled as unreasonable distance	0	0	670	0	0
Labeled as very far	0	0	284	0	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

10.6 seconds

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	1	1	1	0.702306	0	0.983514	0.740461	0.971936	0.0337236
recall	1	1	1	1	0	0.983514	0.8	0.983514	0.0337236
f1-score	1	1	1	0.825123	0	0.983514	0.765025	0.976713	0.0337236
support	4650	2820	8803	670	284	0.983514	17227	17227	0.0337236

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	4650	0	0	0	0
Labeled as far	0	2820	0	0	0
Labeled as nearby	0	0	8803	0	0
Labeled as unreasonable distance	0	0	0	670	0
Labeled as very far	0	0	0	284	0

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence close (Fold 1)

Dependence far (Fold 1)

Dependence nearby (Fold 1)

Dependence unreasonable distance (Fold 1)

Dependence very far (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

8.4 seconds

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	1	0.997161	0.999886	0.988113	0.947183	0.998142	0.986469	0.998144	0.00454643
recall	0.99957	0.996454	1	0.992537	0.947183	0.998142	0.987149	0.998142	0.00454643
f1-score	0.999785	0.996807	0.999943	0.99032	0.947183	0.998142	0.986808	0.998143	0.00454643
support	4650	2820	8803	670	284	0.998142	17227	17227	0.00454643

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	4648	1	1	0	0
Labeled as far	0	2810	0	0	10
Labeled as nearby	0	0	8803	0	0
Labeled as unreasonable distance	0	0	0	665	5
Labeled as very far	0	7	0	8	269

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence close (Fold 1)

Dependence far (Fold 1)

Dependence nearby (Fold 1)

Dependence unreasonable distance (Fold 1)

Dependence very far (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

10.4 seconds

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	0.995041	0.991166	0.99807	0.998478	0.938776	0.995124	0.984306	0.995161	0.0172138
recall	0.992473	0.994681	0.998637	0.979104	0.971831	0.995124	0.987345	0.995124	0.0172138
f1-score	0.993755	0.99292	0.998353	0.988696	0.955017	0.995124	0.985749	0.995133	0.0172138
support	4650	2820	8803	670	284	0.995124	17227	17227	0.0172138

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	4615	18	17	0	0
Labeled as far	11	2805	0	0	4
Labeled as nearby	12	0	8791	0	0
Labeled as unreasonable distance	0	0	0	656	14
Labeled as very far	0	7	0	1	276

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 5
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

5.5 seconds

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	1	1	1	0.997024	1	0.999884	0.999405	0.999884	0.0107606
recall	1	1	1	1	0.992958	0.999884	0.998592	0.999884	0.0107606
f1-score	1	1	1	0.99851	0.996466	0.999884	0.998995	0.999884	0.0107606
support	4650	2820	8803	670	284	0.999884	17227	17227	0.0107606

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	4650	0	0	0	0
Labeled as far	0	2820	0	0	0
Labeled as nearby	0	0	8803	0	0
Labeled as unreasonable distance	0	0	0	670	0
Labeled as very far	0	0	0	2	282

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence close (Fold 1)

Dependence far (Fold 1)

Dependence nearby (Fold 1)

Dependence unreasonable distance (Fold 1)

Dependence very far (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	1

Metric details

	close	far	nearby	unreasonable distance	very far	accuracy	macro avg	weighted avg	logloss
precision	1	0.997161	0.999886	0.988113	0.947183	0.998142	0.986469	0.998144	0.00454643
recall	0.99957	0.996454	1	0.992537	0.947183	0.998142	0.987149	0.998142	0.00454643
f1-score	0.999785	0.996807	0.999943	0.99032	0.947183	0.998142	0.986808	0.998143	0.00454643
support	4650	2820	8803	670	284	0.998142	17227	17227	0.00454643

Confusion matrix

	Predicted as close	Predicted as far	Predicted as nearby	Predicted as unreasonable distance	Predicted as very far
Labeled as close	4648	1	1	0	0
Labeled as far	0	2810	0	0	10
Labeled as nearby	0	0	8803	0	0
Labeled as unreasonable distance	0	0	0	665	5
Labeled as very far	0	7	0	8	269

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back